Like fiber-optic communication and ADSL technologies in fixed networks, spot wireless network technologies such as WLAN and ad hoc network technologies have become widespread in mobile networks in recent years. A “network existing everywhere”, namely a ubiquitous network environment that will make possible network connection anywhere, anytime, and from anything is turning into reality. In the field of communication terminal products, devices, such as network-enabled appliances and terminals with cellular/WLAN dual interface, that have a wide variety of capabilities have been developed. In these circumstances, there is a strong demand for “seamless service” that flexibly switches between various access networks and devices having various capabilities in a ubiquitous network environment to make possible communications to continuously provide service to a user without requiring the user to do anything as the user moves about.
The following three seamless techniques for implementing such seamless service have been proposed. A first technique is called the “network seamless” technique. The “network seamless” technique enables a mobile terminal to seamlessly switch from one network to another as the terminal moves about. The mobile terminal has multiple network interfaces and is capable of flexibly switching between different access networks as it moves. The “network seamless” is sometimes also called the “terminal mobility”.
A second technique is called the “device seamless” technique. The “device seamless” technique enables a user to seamlessly switch from a terminal the user is currently using to another terminal. In this technique, a session being used on one terminal can be continuously used on the other terminal. The “device seamless” technique transfers a session on a terminal to another in this way, hence is also called the “session mobility” technique.
A third technique is called the “content seamless” technique. The “content seamless” technique enables switching between media of different formats or types. In the “content seamless” technique, one can switch from one media to another to display or play back a content according to network bandwidth, device capability, or surroundings. The “content seamless” is sometimes called “media adaptive”.
The three techniques are crucially important in implementing seamless service and their respective unique effects can be obtained independently. These techniques can be used separately or in any combination.
Seamless service to which the device seamless technique is applied (hereinafter the service is referred to as “device switching”) is required in the following situation. For example, a user may come home while watching a streaming movie on a mobile terminal such as a cellphone, enter the living room, and immediately turn on a television set to watch the rest of the movie on its large display. Typical mobile terminals have too small a display to watch movies. The device seamless technique would become far more advantageous if the technique enables seamless switching from one device to a more preferable device that is capable of taking over the service when the device is found near the user as the user moves from one place to another. In view of this advantage of device switching, techniques for implementing such device switching have been developed.
FIG. 14 illustrates a typical configuration of a device switching system. The system shown in its entirety in FIG. 14 includes a mobile node (MN) 2001, which is the sender device used, a correspondent node (CN) 2002, which is a node with which the mobile node 2001 is communicating, the Internet 2003, a group of devices 2004 such as a plasma television set, a stereo player, and a personal computer (PC) to which device switching can be performed, and a personal area network (PAN) 2005 consisting of the MN 2001 and the group of devices 2004. The plasma television, stereo player, and PC constituting the device group 2004 are not limitative; they are illustrative only. The MN 2001 does not need to be a mobile device and is no different from the devices in the group 2004.
The MN 2001 and the group of devices 2004 can use the PAN 2005 to communicate one another within the PAN 2005. The CN 2002 and the MN 2001 are communicating with each other through the Internet 2003. The CN 2002 may be a streaming server, for example, and MN 2001 is receiving a streaming movie distributed from the CN 2002. The network interconnecting the MN 2001 and the CN 2002 dose not need to be the Internet; it may be a Third-Generation Partnership Project (3GPP) network, for example.
FIG. 15 is a block diagram illustrating an internal device configuration of MN 2001 for accomplishing device switching. The MN 2001 includes an application 2101, one or more network interface 2102, an output section which displays and outputs data, an input section 2104 which serves as an interface with an user, an operating system 2105, a media transmitter 2106 which communicates with an external device (not shown) through the operating system 2105, a media receiver 2107, a media buffer 2108 which holds media data received at the media receiver 2107, a decoder section 2109 which decodes media data if the media data is compressed, a signaling section 2110 which controls sessions, a service discovery section 2111 which discovers a service provided by an external switching target device (not shown), and middleware 2112 which controls the media transmitter 2106 to transfer received media data to a switching target device.
As mentioned above, the MN 2001 can be considered as one of the devices in the device group 2004. Switching target devices to which switching can be performed has a configuration similar to that of the MN 2001. However, switching target devices do not need to have an application 2101, an input section 2104, and a media transmitter 2106 if they do not need function as a switching source device.
FIG. 16 shows a message sequence between the MN 2001 and a group of switching target devices 2004; FIG. 17 is a flowchart of the message sequence. The assumption here is that the device group 2004 consists of N devices (device 1 to device N). Although not shown, the MN 2001 has another session with CN 2002. The “user” shown in FIG. 16 is the user who is currently using the MN 2001 and can issue an instruction to the MN 2001 through a user interface provided by the application 2101 running on the MN 2001. The arrows in FIG. 16 represent that a message is sent from the arrows' start points to end points.
The steps shown in FIG. 17 will be detailed below. First, a user attempting to make device switching issues a request to list switching target devices (U1) to the MN 2001 through the input section 2104 of the MN 2001 (S2301). The middleware 2112 receives the instruction and broadcasts a service discovery request (M1) to devices 1 to N in the PAN through the service discovery section 2111 (S2302). When the service discovery sections 2111 of switching target devices receive the service discovery request (M1), the service discovery sections 2111 return service discovery responses (M2) to the MN 2001 (S2303). The service discovery protocol used by the service discovery section 2111 is not limited to a particular one. Any of the existing protocols such as UPnP (Universal Plug and Play) SSDP (Simple Service Discovery Protocol), SDP of Blue-tooth, or SLP (Service Location Protocol) may be used.
The service discovery section 2111 of the MN 2001 receives the service discovery responses (M2) from the switching target devices and notifies the middleware 2112 of the responses (M2). The middleware 2112 generates, on the basis of the notification, a switching target device list consisting of devices capable of providing the service and the output section 2103 presents (U2) the list to the user through the operating system 2105 (S2304). When the user receives the switching target device list, the user selects (U3) a switching target device from the presented switching target device list through the input section 2104 and the selected device is notified to the middleware 2112 (S2305). It is assumed here that “Device 2” is selected.
The middleware 2112 sends a request to establish session and prepare for media processing (M3) to the switching target “Device 2” through the signaling section 2110 in order to establish a session with the selected switching target device, “Device 2”, in the PAN to transmit media data. At the same time, the middleware 2112 directs the media receiver 2107 and the media transmitter 2106 to start preparing for transferring received media data to the switching target device (S2306). The protocol used by the signaling section 2110 is not limited to any particular one; any of existing protocols such as SIP (Session Initiation Protocol) may be used.
When the signaling section 2110 of the switching target, “Device 2”, receives the request to establish session and prepare for media processing from the MN 2001, the middleware 2112 directs the media receiver 2107, the media buffer 2108, and the decoder section 2109 to start preparing for media processing. On completion of the media processing preparation, the middleware 2112 sends a session established and prepared for media processing response (M4) through the signaling section 2110 (S2307).
When the signaling section 2110 of the MN 2001 receives the session established and prepared for media processing response (M4) from the switching target, “Device 2”, the middleware 2112 directs the media transmitter 2106 to start transmission (D1) of the media data that the media receiver 2107 has received from the CN 2002 (S2308).
The media receiver 2107 of the switching target “Device 2” receives the media data transferred from the MN 2001 and starts buffering the media data in the media buffer 2108. After a certain amount of data is buffered in the media buffer 2108, the middleware 2112 directs the decoder section 2109 to start decoding the media data if the data is compressed. The decoder section 2109 outputs (D2) the decoded media data to the output section 2103 through the operating system 2105 (S2309). If the media data is not compressed, the decoder section 2109 outputs the media data buffered in the media buffer 2108 to the output section 2103 through the operating system 2105.
Through the steps described above, device switching from the MN 2001 to Device 2 selected as the switching target device can be made. As a result, the media data that was outputted on the output section 2103 of the MN 2001 is outputted on the output section 2103 of the switching target “Device 2”. In this switching sequence, the time between the user issuing the request to list switching target devices (U1) to the MN 2001 and the switching target “Device 2” outputting data on its outputting section 2103 (D2) is the time required for device switching. The shorter the time required for the switching, the shorter the waiting time experienced by the user involved in the switching and the more seamless the service provided to the user.
However, the conventional technique described above takes time on the order of several seconds between issuing a service discovery request (M1) and receiving a service discovery response (M2). The technique also takes time on the order of several seconds between issuing a request to establish session and prepare media processing (M3) and receiving a session established and prepared for media processing response (M4). Consequently, the time required for the switching, that is, the waiting time experienced by the user is quite long. In these circumstances, if the user is watching a soccer game broadcast in real time, for example, the user can miss an important moment such as a goal scoring moment because of the long device switching time.
Two techniques for reducing the time required for device switching are disclosed in the article “Mobile multimedia middleware for implementing seamless service” in Technical Report of Information Processing Society of Japan (“Mobile computing and wireless communication”, 2001, No. 18-No. 35). The first one of the techniques is to “perform service discovery beforehand at regular intervals” and the second technique is to “establish sessions with all devices found by the service discovery and prepare for media processing beforehand”.
How the two techniques can reduce the time required for device switching will be described with reference to drawings. It should be noted that MN 2001 and switching target devices used in the techniques disclosed in the article “Mobile multimedia middleware for implementing seamless service” in Technical Report of Information Processing Society of Japan (Mobile computing and wireless communication”, 2001, No. 18-No. 35) have the same internal configuration as those of typical, conventional terminals, except for middleware 2112, which operates differently. Therefore, in order to distinguish the middleware in the techniques disclosed in the article “Mobile multimedia middleware for implementing seamless service” in Technical Report of Information Processing Society of Japan (“Mobile computing and wireless communication”, 2001 No. 18-35) from middleware 2112 according to typical, conventional techniques, the former middleware will be referred to as the “fast middleware”. The same elements as those in the conventional techniques will be labeled with the same reference numerals that are used in the foregoing description.
FIGS. 18, 19, and 20 are a diagram showing a message sequence between an MN 2001 and a group of devices 2004 (FIG. 18) in the techniques disclosed in the article “Mobile multimedia middleware for implementing seamless service” in Technical Report of Information Processing Society of Japan (“Mobile computing and wireless communication”, 2001, No. 18-No. 35), a flowchart showing operation of the sender device in the sequence (FIG. 19), and a flowchart showing operation of a switching target device in the sequence (FIG. 20). Again, it is assumed that the device group 2004 consists of N devices, device 1 to device N. Although not shown, the MN 2001 has another session with a CN 2002.
Referring to FIG. 19, operation of the sender device from which switching is to be made will be described first. The sender device determines whether a session with the CN 2002 is retained (S2501). If the sender device determines that a session with the CN 2002 is not retained (S2501: No), the sender device will end the process. On the other hand, if the sender device determines that a session with CN 2002 is retained (S2501: Yes), the fast middleware of the sender device broadcasts a service discovery request (M1) to devices 1 to N in the PAN through the service discovery section 2111 at regular intervals (S2502). When receiving the service discovery request, switching target devices send (return) service discovery responses to the service discovery sections 2111 of the sender device. The sender device provides notification about the service discovery responses received from the switching target devices from the service discovery section 2111 to the fast middleware. The fast middleware generates a list of switching target devices capable of providing the service, on the basis of the notification, and holds the list internally (S2503). If the fast middleware already holds a switching target device list, the fast middleware overwrites the list with the newly generated switching target device list.
The fast middleware sends a request to establish session and prepare for media processing (M3) through the signaling section 2110 in order to establish a session with the candidate switching target devices on the switching target device list to transmit media (S2504). The number of the candidate switching target device is equal to n (where 1≦n≦N). The fast middleware directs the media receiver 2107 and the media transmitter 2106 to start preparing for transferring received media data to a switching target device (S2505). When the signaling section 2110 receives a session established and prepared for media processing response from the candidate switching target devices, the fast middleware waits a predetermined amount of time (S2506). During the waiting time, the fast middleware determines whether a request to list switching target devices (U1) is issued by the user (S2507). If it determines that such a request is not made (S2507: No), the sender device returns to step S2501 and repeats the subsequent service discovery steps. On the other hand, if the fast middleware determines that a list request is issued (S2507: Yes), the process proceeds to step S2508, where the fast middleware presents the switching target device list it internally holds to the user through the output section 2103 (U2).
The user selects a switching target device from the switching target device list through the input section 2104 (U3) (S2509). Information about the device selected here is provided to the fast middleware through the operating system 2105. In the example shown in the sequence diagram, it is assumed that “Device 2” is selected.
The fast middleware directs the media transmitter 2106 to start transmission of the media data (D1) received by the media receiver 2107 from CN 2002 (S2510). In response to this direction, the media transmitter 2106 starts transferring the media data to the switching target “Device 2”. The fast middleware sends a session discontinue request (M5) to all candidate switching target devices other than “Device 2” through the signaling section 2110 (S2511), then receives session discontinued responses from those devices (M6) and ends the switching operation. Up to this point, operation of the sender device has been described.
Operation of a switching target device will be described next with reference to FIG. 20. When a candidate switching target device receives a service discovery request, its service discovery section 2111, sends (returns) a service discovery response (M2) to the MN 2001 (S2601). The signaling section 2110 of the candidate switching target device receives a request to establish a session and prepare media processing (“Session establish request” in FIG. 20) from the MN 2001 (S2602). After receiving the session establish request, the fast middleware directs the media receiver 2107, media buffer 2108, and the decoder section 2109 to start preparing for media processing (S2603). On completion of the preparation for media processing, the fast middleware sends a session established and prepared for media processing response (M4) to the sender device through the signaling section 2110 (S2604).
If media data is transferred, the media receiver 2107 of the switching target “Device 2”, starts to receive the media data transferred from the sender device (S2605) and starts buffering (storing) the media data in the media buffer 2108 (S2606). After a certain amount of data is buffered in the media buffer 2108, the fast middleware determine whether the buffered media data is uncompressed data (S2607). If the media data is uncompressed (S2607: Yes), the fast middleware sends the data buffered in the media buffer 2108 to the output section 2103 through the operating system 2105. On the other hand, if the media data is compressed (S2607: No), the fast middleware directs the decoder section 2109 to start decoding the data (S2608). The decoded data is then sent to the output section 2103 through the operating system 2105. The output section 2103 starts outputting the received data (D2), and thus the switching operation is completed (S2609).
On the other hand, if the signaling section 2110 of the candidate switching target device receives a session discontinue request (M5) from the sender device, the fast middleware directs the media receiver 2107, the media buffer 2108, and the decoder section 2109 to terminate the preparation for media processing. Thus the preparation for media reception will end (S2610). After the preparation for media processing terminates, the fast middleware sends a session discontinued response (M6) to the sender device through the signaling section 2110 (S2611). Thus, the switching operation ends.
In this way, the techniques disclosed in the article “Mobile multimedia middleware for implementing seamless service” in Technical Report of Information Processing Society of Japan (“Mobile computing and wireless communication”, 2001, No. 18-35) enable fast switching from the sender device to the switching target device. The effects of each of the above-described two new techniques for reducing the time required for device switching will be summarized below.
The effect of the first technique (which performs service discovery operation at regular intervals beforehand) is that the time involved in the service discovery at device switching can be practically reduced to zero by performing the operation from transmission of service discovery request (M1) to reception of a service discovery response (M2) (steps S2502 through S2503 and S2601) at regular intervals before a request to list switching target devices is issued by the user (S2507). Thus, the first technique can reduce the waiting time for the device list to be presented to the user.
The effect of the second technique (which establishes sessions with all devices found as a result of device discovery and prepares for media processing beforehand) is that the time involved in the session establishment at device switching can be practically reduced to zero by performing the operation from transmission of a request to establish a session and prepare for media processing (M3) to reception of a session established and prepared for media processing response (M4) (steps S2504 through S2506 and S2602 through S2604) for all candidate switching target devices found through service discovery before a switching target device is selected by the user (S2509). Thus, the second technique can reduce the waiting time for the selected switching target device to become prepared for media processing.
The effects described above significantly reduce the time required for switching, that is, the time between the issuance of a request to list switching target devices (U1) to the sender device by the user attempting to switch and the output of media data on the output section of the switching target device (D2).